Simulation and Virtual Prototyping of Tangible User Interfaces

Prototyping is an important part in research and development of tangible user interfaces (TUIs). On the way from the idea to a working prototype, new hardware prototypes usually have to be crafted repeatedly in numerous iterations. This brings us to think about virtual prototypes that exhibit the same functionality as a real TUI, but reduce the amount of time and resources that have to be spent. Building upon existing open-source software – the middleware Robot Operating System (ROS) and the 3D simulator Gazebo – we have created a toolkit that can be used for developing and testing fully functional implementations of a tangible user interface as a virtual device. The entire interaction between the TUI and other hardware and software components is controlled by the middleware, while the human interaction with the TUI can be explored using the 3D simulator and 3D input/output technologies. We argue that by simulating parts of the hardware-software co-design process, the overall development effort can be reduced.